Apparatus, system, and method for detecting user input via hand gestures and arm movements

ABSTRACT

An artificial-reality system comprising (1) a wearable dimensioned to be donned on a body part of a user, wherein the wearable comprises (A) a set of electrodes that detect one or more neuromuscular signals via the body part of the user and (B) a transmitter that transmits an electromagnetic signal, (2) a head-mounted display communicatively coupled to the wearable, wherein the head-mounted display comprises a set of receivers that receive the electromagnetic signal, and (3) one or more processing devices that (1) determine, based at least in part on the neuromuscular signals, that the user has made a specific gesture and (2) determine, based at least in part on the electromagnetic signal, a position of the body part of the user when the user made the specific gesture. Various other apparatuses, systems, and methods are also disclosed.

INCORPORATION BY REFERENCE

This application is a continuation of U.S. Application No. 17/705,899filed 28 Mar. 2022, the contents of which are incorporated in theirentirety by this reference. This application claims the benefit ofpriority under 35 U.S.C. §119(e) to U.S. Provisional Application No.63/253,667 filed Oct. 8, 2021, the contents of which are incorporatedherein by reference in their entirety.

BRIEF DESCRIPTION OF DRAWINGS

The accompanying drawings illustrate a number of exemplary embodimentsand are parts of the specification. Together with the followingdescription, the drawings demonstrate and explain various principles ofthe instant disclosure.

FIG. 1 is an illustration of an artificial-reality system for detectinguser input via hand gestures and hand movements according to one or moreembodiments of this disclosure.

FIG. 2 is an illustration of an exemplary wearable that facilitatesdetecting user input via hand gestures and hand movements according toone or more embodiments of this disclosure.

FIG. 3 is an illustration of an exemplary head-mounted display thatfacilitates detecting user input via hand gestures and hand movementsaccording to one or more embodiments of this disclosure.

FIG. 4 is an illustration of an exemplary implementation of anartificial-reality system for detecting user input via hand gestures andarm movements according to one or more embodiments of this disclosure.

FIG. 5 is an illustration of an exemplary implementation of anartificial-reality system for detecting user input via hand gestures andarm movements according to one or more embodiments of this disclosure.

FIG. 6 is an illustration of an exemplary implementation of a wearablethat facilitates detecting user input via hand gestures and armmovements according to one or more embodiments of this disclosure.

FIG. 7 is an illustration of exemplary neuromuscular signals detected bya wearable in connection with user input entered via hand gestures andarm movements according to one or more embodiments of this disclosure.

FIG. 8 is an illustration of an exemplary angle-of-arrival calculationfor determining user input entered via hand gestures and arm movementsaccording to one or more embodiments of this disclosure.

FIG. 9 is an illustration of an exemplary point-of-view implementationof an artificial-reality system for detecting user input via handgestures and arm movements according to one or more embodiments of thisdisclosure.

FIG. 10 is an illustration of an exemplary spherical coordinate systemfor translating user input to a display screen of a head-mounted displayaccording to one or more embodiments of this disclosure.

FIG. 11 is a flowchart of an exemplary method for detecting user inputvia hand gestures and arm movements according to one or more embodimentsof this disclosure.

FIG. 12 is an illustration of exemplary augmented-reality glasses thatmay be used in connection with embodiments of this disclosure.

FIG. 13 is an illustration of an exemplary virtual-reality headset thatmay be used in connection with embodiments of this disclosure.

FIG. 14 is an illustration of exemplary haptic devices that may be usedin connection with embodiments of this disclosure.

FIG. 15 is an illustration of an exemplary virtual-reality environmentaccording to embodiments of this disclosure.

FIG. 16 is an illustration of an exemplary augmented-reality environmentaccording to embodiments of this disclosure.

FIGS. 17A and 17B are illustrations of an exemplary human-machineinterface configured to be worn around a user’s lower arm or wrist.

FIGS. 18A and 18B are illustrations of an exemplary schematic diagramwith internal components of a wearable system.

While the exemplary embodiments described herein are susceptible tovarious modifications and alternative forms, specific embodiments havebeen shown by way of example in the drawings and will be described indetail herein. However, the exemplary embodiments described herein arenot intended to be limited to the particular forms disclosed. Rather,the instant disclosure covers all modifications, combinations,equivalents, and alternatives falling within this disclosure.

DETAILED DESCRIPTION

The present disclosure is generally directed to apparatuses, systems,and methods for detecting user input via hand gestures and armmovements. As will be explained in greater detail below, theseapparatuses, systems, and methods may provide numerous features andbenefits.

Artificial reality often provides a rich, immersive experience in whichusers are able to interact with virtual objects and/or environments inone way or another. In this context, artificial reality may constitute aform of reality that has been altered by virtual objects forpresentation to a user. Such artificial reality may include and/orrepresent virtual reality, augmented reality, mixed reality, hybridreality, or some combination and/or variation of one or more of thesame.

Although artificial-reality systems are commonly implemented for gamingand other entertainment purposes, such systems are also implemented forpurposes outside of recreation. For example, governments may use themfor military training simulations, doctors may use them to practicesurgery, engineers may use them as visualization aids, and co-workersmay use them to facilitate inter-personal interactions and collaborationfrom across the globe.

Certain artificial-reality systems may incorporate hands-on controllersthat enable users to enter input capable of modifying their artificialreality experiences. Unfortunately, these hands-on controllers may limitthe users’ mobility and/or movements, especially hand-based actionsand/or gestures. To resolve these limitations, some artificial-realitysystems may incorporate wearables capable of sensing a few motions,actions, and/or gestures made by users. The sensing of other motions,actions, and/or gestures, however, may prove challenging and/orimpracticable via such wearables.

For example, some wearables may be unable to accurately detect and/ortrack the distance and/or location of a body part positioned proximateto a wearable. Additionally or alternatively, some wearables may beunable to translate the distance and/or location of such a body partinto a virtual component presented to the user via a head-mounteddisplay (e.g., augmented-reality glasses), much less controlling thehead-mounted display via such a body part. The instant disclosure,therefore, identifies and addresses a need for additional systems andmethods for detecting user input via hand gestures and arm movements.

As will be described in greater detail below, an artificial-realitysystem may include and/or represent a wearable (e.g., a wristband and/orwatch) and/or a head-mounted display (e.g., augmented-reality glasses)that are communicatively coupled to one another. In one example, thewearable may be dimensioned to be donned on a body part (e.g., a wrist)of a user of the artificial reality system. In this example, thewearable may include and/or represent (1) a set of electrodes (e.g., EMGsensors) that detect one or more neuromuscular signals via the body partof the user and (2) a transmitter (e.g., an ultra-wideband radio) thattransmits an electromagnetic signal.

In some examples, the head-mounted display may include and/or representa set of receivers that receive the electromagnetic signal transmittedby the transmitter included on the wearable. In such examples, theartificial-reality system may include and/or represent one or moreprocessing devices that (1) determine, based at least in part on theneuromuscular signals detected via the body part of the user, that theuser has made a specific gesture and (2) determine, based at least inpart on the electromagnetic signal received by the set of receiversincluded on the head-mounted display, a position of the body part of theuser when the user made the specific gesture.

In some examples, at least one of the processing devices may beincorporated in the wearable. Additionally or alternatively, at leastone of the processing devices may be incorporated in the head-mounteddisplay.

In some examples, the user’s hand gestures and/or arm movements mayserve and/or operate as a user interface for the artificial-realitysystem. For example, the user’s hand gestures and/or arm movements maygenerate control signals that are translated into commands for theartificial-reality system.

In some examples, the wearable may include and/or represent EMG sensorsthat detect and/or measure muscle activity and/or patterns. In oneexample, the artificial-reality system may include and/or representBluetooth radios that facilitate configuring and/or pairing the wearableand/or head-mounted display. Additionally or alternatively, theBluetooth radios may transmit EMG data from the wearable to thehead-mounted display.

In some examples, the wearable may include and/or represent one or moreultra-wideband impulse radios that provide and/or transmit precisetimestamped impulse signals to the head-mounted display forangle-of-arrival calculations. In such examples, the head-mounteddisplay may include and/or represent an array of ultra-wideband antennas(e.g., 2, 3, 4, or more antennas) that receive the impulse signals. Thehead-mounted display may identify and/or detect the times of arrival forthe impulse signals as received by the ultra-wideband antennas. Thehead-mounted display may then calculate and/or compute the varyingtravel times of the impulse signals relative to the array ofultra-wideband antennas based at least in part on the times of arrivaland the timestamps. In one example, the head-mounted display may thenconvert the times of arrival and/or the travel times for the impulsesignals to an angle of arrival of the impulse signals, which correspondto and/or represent the location (e.g., a 2-dimensional and/or3-dimensional representation) of the wearable within a defined field ofview of the head-mounted display. In this example, the accuracy and/orprecision of the angle of arrival may increase with the number ofantennas included in the array. The head-mounted display may thengenerate a pointer and/or superimpose the same on top of anaugmented-reality presentation provided to and/or viewed by the user.

In some examples, an ultra-wideband impulse radio incorporated in thewearable may wirelessly transmit precise timestamp data and/or EMGsignal data to the array of ultra-wideband antennas incorporated in thehead-mounted display. In these examples, if the wearable is locatedwithin the field of view of the head-mounted display, the head-mounteddisplay may activate and/or generate a pointer and/or cursor for displayand/or presentation to the user. In one example, the head-mounteddisplay may determine hand gestures performed by the user based at leastin part on the EMG signal data. In this example, the hand gestures maycorrespond to and/or represent commands and/or computer-readableinstructions for the artificial-reality system.

In some examples, the head-mounted display may determine the proper2-dimensional and/or 3-dimensional location or position of the pointerand/or cursor within the field of view based at least in part on theangle of arrival. By combining the EMG signal data and theangle-of-arrival data, the head-mounted display may be able to create acontrol mechanism and/or user interface that enables the user to controland/or interact with the virtual features displayed to the user withouttouching the head-mounted display or even the wearable.

The following will provide, with reference to FIGS. 1-10 , detaileddescriptions of exemplary devices, systems, components, andcorresponding implementations for detecting user input via hand gesturesand arm movements. In addition, detailed descriptions of methods fordetecting user input via hand gestures and arm movements in connectionwith FIG. 11 . The discussion corresponding to FIGS. 12-18 will providedetailed descriptions of types of exemplary artificial-reality devices,wearables, and/or associated systems that may support and/or contributeto detecting user input via hand gestures and arm movements.

FIG. 1 illustrates an exemplary artificial-reality system 100 thatincludes and/or represents a wearable 102 and/or a head-mounted display104 capable of communicating with one another. As illustrated in FIG. 1, wearable 102 may include and/or represent a processor 120(1), a radio112(1), a set of electrodes 116, and/or a transmitter 114. In someexamples, head-mounted display 104 may include and/or represent aprocessor 120(2), a radio 112(2), a set of receivers 118, a camera 128,a display screen 110, and/or a power source 122.

In some examples, wearable 102 may refer to and/or represent any type orform of computing device that is worn as part of an article of clothing,an accessory, and/or an implant. In one example, wearable 102 mayinclude and/or represent a wristband secured to and/or worn by the wristof a user. Additional examples of wearable 102 include, withoutlimitation, armbands, pendants, bracelets, rings, jewelry, ankle bands,clothing, smartwatches, electronic textiles, shoes, clips, headbands,gloves, variations or combinations of one or more of the same, and/orany other suitable wearable devices.

In some examples, head-mounted display 104 may refer to and/or representany type of display and/or visual device that is worn on and/or mountedto a user’s head or face. In one example, head-mounted display 104 mayinclude and/or represent a pair of augmented reality (AR) glassesdesigned to be worn on and/or secured to a user’s head or face. Asillustrated in FIG. 3 , head-mounted display 104 may include and/orincorporate display screen 110 as lenses and/or corresponding partiallysee-through components on such AR glasses. In this example, head-mounteddisplay 104 may include and/or incorporate cameras 128(1) and 128(2)directed and/or aimed toward the user’s line of sight and/or field ofview. In another example, head-mounted display 104 may include and/orrepresent a virtual-reality headset and/or any other suitable type orform of artificial-reality headset.

In some examples, wearable 102 and/or head-mounted display 104 mayachieve and/or establish one or more links, connections, and/or channelsof communication with one another. For example, wearable 102 andhead-mounted display 104 may be able to communicate with one another viatransmitter 114 and receivers 118, respectively. In this example,transmitter 114 and receivers 118 may achieve, support, facilitate,and/or establish ultra-wideband impulse radio (UWB-IR) communication 140between wearable 102 and head-mounted display 104. Additionally oralternatively, wearable 102 and head-mounted display 104 may be able tocommunicate with one another via radio 112(1) and radio 112(2),respectively. In this example, radio 112(1) and radio 112(2) mayachieve, support, facilitate, and/or establish radio communication 138(e.g., Bluetooth communication) between wearable 102 and head-mounteddisplay 104.

In some examples, electrodes 116 may each constitute and/or representany type or form of electrical conductor capable of detecting and/orsensing neuromuscular signals via a user’s body. In one example,electrodes 116 may include and/or represent neuromuscular sensors and/orelectromyography (EMG) sensors arranged, configured, and/or disposedcircumferentially around wearable 102 as illustrated in FIG. 2 .Additional examples of electrodes 116 include, without limitation,mechanomyography (MMG) sensors, sonomyography (SMG) sensors,combinations or variations of one or more of the same, and/or any othersuitable electrodes. Any suitable number and/or arrangement ofelectrodes 116 may be applied to wearable 102.

In some examples, electrodes 116 may be communicatively coupled to oneanother and/or to processor 120(1) by flexible electronics, connectors,traces, and/or wiring. Additionally or alternatively, electrodes 116 maybe integrated with and/or into an elastic band and/or wristband ofwearable 102.

In some examples, electrodes 116 may be arranged in a specific and/ordeliberate configuration across wearable 102. In one example, electrodes116 may be separated and/or spaced from one another along wearable 102by one or more known distances.

In some embodiments, the output of one or more of electrodes 116 may beprocessed, amplified, rectified, and/or filtered via hardware signalprocessing circuitry. Additionally or alternatively, the output of oneor more of electrodes 116 may be processed, amplified, rectified, and/orfiltered via signal processing software or firmware. Accordingly, theprocessing of neuromuscular signals may be performed in hardware,software, and/or firmware.

In some examples, one or more of processors 120(1) and 120(2) mayinclude and/or represent any type or form of hardware-implementedprocessing device capable of interpreting and/or executingcomputer-readable instructions. In one example, processor 120(1) or120(2) may access and/or modify certain software modules to facilitateand/or support detecting user input via hand gestures and/or armmovements. Examples of processors 120(1) and 120(2) include, withoutlimitation, physical processors, Central Processing Units (CPUs),microprocessors, microcontrollers, Field-Programmable Gate Arrays(FPGAs) that implement softcore processors, Application-SpecificIntegrated Circuits (ASICs), portions of one or more of the same,variations or combinations of one or more of the same, and/or any othersuitable processing device.

In some examples, wearable 102 may include and/or incorporate a wearableband. For example, wearable 102 may include and/or represent a strapand/or band designed and/or dimensioned to at least partially encompassthe user’s wrist and/or arm. The strap and/or band may include and/orcontain a variety of different materials. Examples of such materialsinclude, without limitation, cottons, polyesters, nylons, elastics,plastics, neoprene, rubbers, metals, woods, composites, combinations orvariations of one or more of the same, and/or any other suitablematerials. The strap and/or band may be defined and/or formed in avariety of shapes and/or sizes with the aim of securing wearable 102 tothe user’s wrist and/or arm. In one example, the strap and/or band mayinclude and/or represent one or more segments, links, and/or sections.Additionally or alternatively, the strap and/or band may be adjustableto provide a one-size-fits-most feature.

In some examples, wearable 102 and/or head-mounted display 104 mayinclude and/or represent one or more additional components, devices,and/or mechanisms that are not necessarily illustrated and/or labelledin FIG. 1 . For example, wearable 102 and/or head-mounted display 104may include and/or represent one or more memory devices that are notnecessarily illustrated and/or labelled in FIG. 1 . Such memory devicesmay include and/or store computer-executable instructions that, whenexecuted by processor 120(1) or 120(2), cause processor 120(1) or 120(2)to perform one or more tasks directed to detecting user input via handgestures and arm movements. Additionally or alternatively, although notnecessarily illustrated and/or labelled in this way in FIG. 1 , wearable102 and/or head-mounted display 104 may include and/or representcircuitry, transistors, resistors, capacitors, diodes, transceivers,sockets, wiring, circuit boards, additional processors, and/oradditional memory devices, batteries, cabling, and/or connectors, amongother components.

In some examples, such memory devices may include and/or represent anytype or form of volatile or non-volatile storage device or mediumcapable of storing data and/or computer-readable instructions. In oneexample, such memory devices may store, load, and/or maintain one ormore modules and/or trained inferential models that perform certaintasks, classifications, and/or determinations in connection withlocalizing motor unit action potential to facilitate spike decompositionand stable representation. Examples of memory 108 include, withoutlimitation, Random Access Memory (RAM), Read Only Memory (ROM), flashmemory, Hard Disk Drives (HDDs), Solid-State Drives (SSDs), optical diskdrives, caches, variations or combinations of one or more of the same,and/or any other suitable storage memory.

In some, wearable 102 and/or head-mounted display 104 may exclude and/oromit one or more of the components, devices, and/or mechanisms that areillustrated and/or labelled in FIG. 1 . For example, wearable 102 and/orhead-mounted display 104 may exclude and/or omit radio 112(1) or radio112(2), respectively. In this example, UWB-IR communication 140 betweenwearable 102 and head-mounted display 104 may still be available and/orprovided via transmitter 114 and receivers 118.

In some examples, when wearable 102 is worn by the user, electrodes 116may interface and/or make physical contact with the user’s skin. In oneexample, wearable 102 may be communicatively coupled to a computingsystem (such as a virtual reality headset, an augmented reality headset,a laptop, a desktop, a smart television, a monitor, etc.). In thisexample, the user may put and/or place his or her body in a certainstate and/or condition (such as a hand gesture) to control and/or modifythe presentation or performance of the computing system. As the userputs and/or places his or her body in that state and/or condition, theuser’s body may generate and/or produce neuromuscular signalsrepresentative, indicative, and/or suggestive of that state orcondition.

In some examples, the neuromuscular signals may traverse and/or travelthrough the user’s body. For example, the user may make a pose and/orgesture that generates neuromuscular signals that traverse down his orher arm toward the hand. In one example, one or more of electrodes 116may detect and/or sense the neuromuscular signals as they traverse downthe arm toward the hand. Electrical conductors (e.g., wiring and/ortraces) coupled between those electrodes and processor 120(1) may carryand/or pass such signals and/or their derivatives to processor 120(1).Processor 120(1) may then generate and/or produce data representative ofthose signals.

In some examples, the data representative of those signals may undergocertain processing and/or conversions. Examples of such data include,without limitation, raw data produced and/or output by electrodes,digital conversions and/or representations of analog signals output byelectrodes, processed digital representations of signals output byelectrodes, combinations or variations of one or more of the same,and/or any other suitable version of data representative ofneuromuscular signals.

In this example, processor 120(1) or 120(2) may analyze and/or evaluatethe data representative of the neuromuscular signals to localize motorunit action potential and/or facilitate spike decomposition or stablerepresentation. For example, processor 120(1) or 120(2) may executeand/or implement one or more software models and/or trained inferentialmodels or classifiers. Processor 120(1) or 120(2) may input and/or feedthe data representative of the neuromuscular signals to one or more ofthe software modules and/or inferential models. From that data, suchsoftware modules and/or inferential models may be able to output and/orproduce a classification that identifies and/or indicates one or moremotor units responsible for certain spikes in the neuromuscular signals.Additionally or alternatively, such software modules and/or inferentialmodels may be able to determine, based at least in part on those motorunits, that the user has made a specific gesture with at least one partof the user’s body (using, e.g., a K-Nearest Neighbors (KNN)classifier).

In some examples, radios 112(1) and 112(2) may each include and/orrepresent a Bluetooth radio and/or a Bluetooth Low Energy radio.Additionally or alternatively, transmitter 114 and/or receivers 118 mayeach include and/or represent one or more UWB-IR devices. In oneexample, transmitter 114 and/or receivers 118 may each be included inand/or represent part of a transceiver that facilitates and/or supportsUWB-IR communications, links, and/or channels. Additional examples oftransmitter 114, receivers 118, and/or radios 112(1)-(2) include,without limitation, WiFi devices, cellular communication devices,Bluetooth radios, Bluetooth Low Energy radios, UWB devices, impulseradios, combinations or variations of one or more of the same, and/orany other suitable wireless communications devices.

In some examples, wearable 102 and head-mounted display 104 may exchangeconfiguration and/or synchronization data with one another via radios112(1) and 112(2). For example, wearable 102 may send and/or transmitconfiguration and/or synchronization data to head-mounted display 104via radios 112(1) and 112(2). Additionally or alternatively,head-mounted display 104 may send and/or transmit configuration and/orsynchronization data to wearable 102 via radios 112(2) and 112(1). Inthese examples, processor 120(1) and/or processor 120(2) may use theconfiguration and/or synchronization data to configure wearable 102and/or head-mounted display 104, respectively, and/or to synchronizewearable 102 and/or head-mounted display 104 to one another. In anotherexample, wearable 102 may send and/or transmit data about theneuromuscular signals detected via the body part of the user tohead-mounted display 104 via radios 112(1) and 112(2).

In some examples, wearable 102 may be dimensioned to be donned on a bodypart of a user. In such examples, electrodes 116 included on wearable102 may detect, sense, and/or conduct one or more neuromuscular signalsvia the body part of the user. In one example, transmitter 114incorporated into wearable 102 may emit, send, and/or transmit anelectromagnetic signal (e.g., a UWB-IR signal) to receivers 118incorporated into head-mounted display 104. In this example, receivers118 incorporated into head-mounted display 104 may receive and/or detectthe electromagnetic signal transmitted by transmitter 114.

In some examples, processor 120(1) or 120(2) may determine that the userhas made a specific gesture and/or arm movement based at least in parton the neuromuscular signals detected via the body part of the user. Inone example, the specific gesture made by the user may include and/orrepresent a pinching action and/or pose. In this example, the pinchingaction and/or pose may involve one of the user’s fingers (e.g., index,middle, ring, and/or pinky fingers) pressing and/or holding against theuser’s thumb. Additional examples of such gestures include, withoutlimitation, fist actions or poses, wrist actions or motions, open-handactions or poses, alternative pinches or poses, hand or arm motions,combinations or variations of one or more of the same, and/or any othersuitable gestures.

In some examples, certain gestures and/or motions may be mapped todifferent input commands for head-mounted display 104. In one example, agesture may be mapped to a certain input command such that, when theuser executes and/or performs the gesture, wearable 102 or head-mounteddisplay 104 causes an application running on head-mounted display 104 toclick, select, and/or modify one or more features (such as virtualcomponents presented by head-mounted display 104). Additionally oralternatively, the input command may be triggered and/or initiated inresponse to the user holding and/or performing the gesture for apredetermined duration.

In some examples, processor 120(2) may determine a position of the bodypart of the user when the user made the specific gesture based at leastin part on the electromagnetic signal received by receivers 118. Forexample, as a UWB-IR device, transmitter 114 may apply and/orincorporate a time stamp into a UWB-IR signal transmitted to receivers118. In this example, the UWB-IR signal may reach and/or arrive atreceivers 118 at different times relative to one another.

In some examples, processor 120(2) may identify and/or determine a firsttime of arrival for the UWB-IR signal as the UWB-IR signal reachesand/or arrives at a first receiver included in receivers 118. In suchexamples, processor 120(2) may identify and/or determine a second timeof arrival for the UWB-IR signal as the UWB-IR signal reaches and/orarrives at a second receiver included in receivers 118. Accordingly, thefirst and second times of arrival may reference and/or correspond to thefirst and second receivers, respectively.

In one example, processor 120(2) may calculate and/or compute an angleof arrival for the UWB-IR signal relative to receivers 118 based atleast in part on the first and second times of arrival and the timestamp. For example, processor 120(2) may subtract the time identified inthe time stamp from the varying times of arrival for the UWB-IR signalas received by receivers 118. The resolution and/or accuracy of thiscalculation may increase and/or improve with the number of receiversinvolved. Accordingly, processor 120(2) may calculate and/or compute amore accurate and/or precise angle of arrival for the UWB-IR signalrelative to receivers 118 by accounting for the times of arrivalrelative to 3 or 4 different receivers incorporated into head-mounteddisplay 104.

In some examples, processor 120(2) may calculate and/or compute at leastone dimension for a position of a virtual component within the field ofview of head-mounted display 104 based at least in part on the angle ofarrival. In one example, processor 120(2) may present and/or display thevirtual component at the position within the field of view of thehead-mounted display 104 based at least in part on the dimension. Forexample, processor 120(2) may cause and/or direct head-mounted display104 to present a cursor and/or pointer at a certain position and/orlocation within the field of view of display screen 110 based at leastin part of the dimension. Additionally or alternatively, processor120(2) may cause and/or direct head-mounted display 104 to superimposeand/or overlay the cursor and/or pointer on or atop display screen 110according to the dimension. In certain examples, processor 120(2) mayalso cause and/or direct head-mounted display 104 to select, open,and/or modify another virtual component presented proximate to thecursor and/or pointer on display screen 110 to account for one or moregestures made by the user.

In some examples, the dimension calculated and/or computed for theposition of the virtual component may constitute and/or represent anazimuth, an elevation, and/or a depth for the virtual component to bepresented within the field of view of head-mounted display. Additionallyor alternatively, processor 120(2) may determine a size, an orientation,and/or a shape of the virtual component based at least in part on thedimension and/or the position.

In some examples, processor 120(2) may detect, sense, and/or determine afirst phase of the UWB-IR signal as the UWB-IR signal reaches and/orarrives at the first receiver included in receivers 118. In suchexamples, processor 120(2) may detect, sense, and/or determine a secondphase of the UWB-IR signal as the UWB-IR signal reaches and/or arrivesat the second receiver included in receivers 118. Accordingly, the firstand second phases of the UWB-IR signal may reference and/or correspondto the first and second receivers, respectively. In one example, thefirst and second phases of the UWB-IR signal may reference and/or berelative to one another.

In one example, processor 120(2) may calculate and/or compute an angleof arrival for the UWB-IR signal relative to receivers 118 based atleast in part on the difference between the first and second phases ofthe UWB-IR signal and/or the time stamp. The resolution and/or accuracyof this calculation may increase and/or improve with the number ofreceivers involved. Accordingly, processor 120(2) may calculate and/orcompute a more accurate and/or precise angle of arrival for the UWB-IRsignal relative to receivers 118 by accounting for the phases of theUWB-IR signal relative to 3 or 4 different receivers incorporated intohead-mounted display 104.

In some examples, processor 120(2) may calculate and/or compute atwo-dimensional (2D) and/or three-dimensional (3D) position of a virtualcomponent within the field of view of head-mounted display 104 based atleast in part on the angle of arrival and/or the phase of the UWB-IRsignal. In one example, processor 120(2) may present and/or display thevirtual component at the 2D and/or 3D position within the field of viewof the head-mounted display 104. For example, processor 120(2) may causeand/or direct head-mounted display 104 to present a cursor and/orpointer at the 2D and/or 3D position within the field of view of displayscreen 110. Additionally or alternatively, processor 120(2) may causeand/or direct head-mounted display 104 to superimpose and/or overlay thecursor and/or pointer on or atop display screen 110 at the 2D and/or 3Dposition.

In some examples, the 2D and/or 3D position of the virtual component mayinclude, involve, and/or account for an azimuth, an elevation, and/or adepth of the virtual component as presented within the field of view ofhead-mounted display 104. Additionally or alternatively, processor120(2) may determine a size, an orientation, and/or a shape of thevirtual component based at least in part on the 2D and/or 3D position.

In some examples, the user may move wearable 102 and/or a correspondingbody part from the field of view of head-mounted display 104. In oneexample, processor 120(2) may detect, sense, and/or determine thatwearable 102 is no longer visible within the field of view of thehead-mounted display 104. In response to this determination, processor120(2) may remove the virtual component representing and/orcorresponding to wearable 102 or the body part from the field of view ofthe head-mounted display 104. For example, processor 120(2) may causeand/or direct head-mounted display 104 to make the cursor and/or pointerdisappear from the field of view of display screen 110.

In some examples, the user may don and/or wear multiple instances ofwearable 102, and all these instances of wearable 102 may becommunicatively coupled to head-mounted display 104. For example, theuser may don and/or wear one wristband on the right wrist and anotherwristband on the left wrist. In this example, the right wristband andthe left wristband may both transmit UWB-IR signals to head-mounteddisplay 104.

In some examples, all instances of wearable 102 may perform any of theoperations and/or functions described above in connection with FIG. 1 .For example, the right wristband may detect and/or sense neuromuscularsignals via the right wrist, and the left wristband may detect and/orsense neuromuscular signals via the left wrist. In this example,head-mounted display 104 may determine that the user made one gesturewith the right hand based on the neuromuscular signals detected via theright wrist. Similarly, head-mounted display 104 may determine that theuser made another gesture with the left hand based on the neuromuscularsignals detected via the left wrist.

In some examples, head-mounted display 104 may determine a position ofthe right wristband and/or a corresponding body part when the userperformed the gesture with the right hand based at least in part on theUWB-IR signal transmitted by the right wristband. Similarly,head-mounted display 104 may determine a position of the left wristbandand/or a corresponding body part when the user performed the gesturewith the left hand based at least in part on the UWB-IR signaltransmitted by the left wristband.

In some examples, head-mounted display 104 may calculate and/or computepositions of virtual components within the field of view of displayscreen 110 based at least in part on the angles of arrival of the UWB-IRsignals transmitted by the right wristband and left wristband. In oneexample, head-mounted display 104 may present and/or display the virtualcomponents at those positions within the field of view of display screen110. For example, head-mounted display 104 may cause and/or directdisplay screen 110 to superimpose and/or overlay a cursor and/or pointercorresponding to the right hand and another cursor and/or pointercorresponding to the left hand. In certain examples, head-mounteddisplay 104 may also cause and/or direct one or more of those cursors orpointers to select, open, and/or modify another virtual componentpresented on display screen 110 to account for one or more gestures madeby the user’s right hand and/or left hand. Those cursors and/or pointersmay appear and/or be presented on display screen 110 simultaneously.

FIG. 4 is an illustration of an exemplary implementation 400 ofartificial-reality system 100 for detecting user input via hand gesturesand arm movements. In some examples, implementation 400 may includeand/or involve a user 410 donning and/or operating wearable 102. In suchexamples, wearable 102 may localize motor unit action potential tofacilitate spike decomposition and stable representation. In oneexample, wearable 102 may detect and/or sense neuromuscular signals 440traversing the body of user 410 via electrodes 116. Wearable 102 maythen translate the neuromuscular signals from a time-domainrepresentation into a frequency-domain representation and/or aspatial-domain representation for further processing.

In some examples, wearable 102 may identify, within the portion of thebody of user 410, a motor unit responsible for the spike inneuromuscular signals 440 by decomposing the spike in neuromuscularsignals 440. In one example, a motor unit may include and/or represent amotor neuron and/or skeletal muscle fibers that are innervated by themotor neuron’s axonal terminals. For example, a motor unit may includeand/or represent a motor neuron along with all the muscle fibersstimulated by the motor neuron.

In some examples, wearable 102 may determine that the user has made aspecific gesture with at least one body part based at least in part onthe motor unit responsible for the spike in neuromuscular signals 440.For example, wearable 102 may process neuromuscular signals 440 astranslated into the frequency-domain representation via amachine-learning classifier (e.g., a KNN classifier). In this example,wearable 102 may detect a spike pattern indicative of the specificgesture via the machine-learning classifier and then determine that theuser made the specific gesture based at least in part on the spikepattern. Additionally or alternatively, wearable 102 may then directhead-mounted display 104 to manipulate and/or alter one or more audioand/or visual elements presented via head-mounted display 104 to accountfor the specific gesture made by the user.

FIG. 5 illustrates an exemplary implementation 500 of artificial-realitysystem 100 for detecting user input via hand gestures and arm movements.In some examples, implementation 500 may include and/or involve a userdonning and/or operating wearable 102. In such examples, wearable 102may detect and/or sense neuromuscular signals 440 traversing the body ofthe user via electrodes 116. In one example, wearable 102 may thentranslate neuromuscular signals 440 from a time-domain representationinto a frequency-domain representation to facilitate and/or supportdetecting and/or identifying gestures 502 and/or motion 504.

In some examples, wearable 102 may detect and/or identify one or more ofgestures 502 and/or motion 504 based at least in part on neuromuscularsignals 440. In such examples, wearable 102 may send and/or transmitinformation or data indicating that the user performed one or more ofgestures 502 and/or motion 504 to head-mounted display 104 via radiocommunication 138 and/or UWB-IR communication 140.

In other examples, wearable 102 may send and/or transmit information ordata representative of neuromuscular signals 440 to head-mounted display104 via radio communication 138 or UWB-IR communication 140. In suchexamples, head-mounted display 104 may detect and/or identify one ormore of gestures 502 and/or motion 504 based at least in part on theinformation or data representative of neuromuscular signals 440.

In some examples, processor 120(1) and/or transmitter 114 incorporatedin wearable 102 may tag a UWB-IR signal with a time stamp. In suchexamples, transmitter 114 may send and/or transmit the tagged UWB-IRsignal to the set of receivers 118 incorporated in head-mounted display104 via UWB-IR communication 140. For example, the tagged UWB-IR signalmay reach and/or arrive at both receivers 118(1) and 118(2), which arepositioned at a distance 510 from one another. In this example,head-mounted display 104 may calculate and/or compute the angle ofarrival and/or the phase difference of arrival for the tagged UWB-IRsignal relative to receivers 118(1) and 118(2) based at least in part onthe time stamp.

In some examples, head-mounted display 104 may perform anangle-of-arrival calculation 800 in FIG. 8 to determine and/or estimatea position for a virtual component (e.g., a pointer and/or cursor)superimposed over display screen 110. As a specific example,head-mounted display 104 may calculate and/or compute the RF carrierphase difference by applying the following formula:

$\text{Δφ} = \text{φ}_{Rx118{(1)}} - \text{φ}_{Rx118{(2)}} = 2\pi( {f \times \text{Δ}t} ) = 2\pi( {f \times \frac{\text{Δ}D}{c}} ) =$

$2\pi( \frac{d \times \cos(\theta)}{\lambda} ),$

where φ_(Rx118(1)) represents the phase of the carrier signal relativeto receiver 118(1), φ_(Rx118(2)) represents the phase of the carriersignal relative to receiver 118(2), f represents the frequency of thecarrier signal, Δt represents the travel time or time of arrival for thecarrier signal between transmitter 114 and the corresponding receiver,ΔD represents the distance between transmitter 114 and the correspondingreceiver, c represents the speed of light, d represents the knowndistance between receivers 118(1) and 118(2), θ represents the angle ofarrival of the carrier signal relative to receivers 118(1) and 118(2),and λ represents the wavelength of the carrier signal. Additionally oralternatively, head-mounted display 104 may calculate and/or compute theangle of arrival of the carrier signal relative to receivers 118(1) and118(2) by applying the following formula:

$\theta = acos( \frac{\text{Δφ} \times \lambda}{2\pi \times d} ).$

In one example, the angle of arrival of the carrier signal mayconstitute and/or represent the direction from which the carrier isreceived relative to receivers 118(1) and 118(2).

In some examples, φ and θ may correspond to and/or represent the azimuthand elevation, respectively, of wearable 102 and/or the virtualcomponent within the field of view. For example, the φ calculation maytranslate and/or convert to the azimuth of the virtual component (e.g.,a pointer and/or cursor) to be presented within a spherical coordinatesystem of head-mounted display 104. In this example, the θ calculationmay translate and/or convert to the elevation of the virtual componentto be presented within a spherical coordinate system of head-mounteddisplay 104.

In some examples, head-mounted display 104 may calculate and/or computethe travel time and/or the time of arrival for the carrier signalbetween transmitter 114 and receivers 118(1) and 118(2). For example,processor 120(2) incorporated in head-mounted display 104 may identifyand/or detect the time stamp tagged to the UWB-IR signal. In thisexample, processor 120(2) may also identify and/or detect the times ofarrival of the carrier signal as received by receiver 118(1) and/orreceiver 118(2). In one example, processor 120(2) may determine traveltimes and/or time deltas for the carrier signal by subtracting the timeidentified in the time stamp from the times of arrival. Processor 120(2)may then calculate and/or compute the angle of arrival for the carriersignal relative to receivers 118(1) and 118(2) based at least in part onthe travel times and/or time deltas.

In some examples, the angle of arrival for the carrier signal maycorrespond to and/or represent the location (e.g., a 2D and/or 3Drepresentation) of the wearable and/or an associated body part withinthe field of view of head-mounted display 104. In such examples,head-mounted display 104 may generate a virtual component (e.g., apointer and/or cursor) that represents the wearable and/or theassociated body for presentation and/or overlay on display screen 110 ofhead-mounted display 104. Head-mounted display 104 may convert and/ortranslate the angle of arrival for the carrier signal into a location(represented, e.g., as an azimuth and/or an elevation) for thecorresponding coordinate system and/or grid of display screen 110. Inone example, the virtual component may be visually combined (in, e.g.,an augmented-reality environment or scenario) with one or more realcomponents that are visible through display screen 110.

FIG. 6 illustrates an exemplary implementation 600 of artificial-realitysystem 100 for detecting user input via hand gestures and arm movements.In some examples, implementation 600 may include and/or involve a userdonning and/or operating wearable 102 on skin 602. In such examples,wearable 102 may include and/or represent an electrode 606 coupled toskin 602. In one example, an EMG signal 612 may traverse and/or travelthrough muscles 604 of the user’s body.

In some examples, electrode 606 may detect and/or sense EMG signal 612traversing muscles 604 of the user’s body via skin 602. In one example,wearable 102 and/or head-mounted display 104 may then translate and/orconvert EMG signal 612 from a time-domain representation into afrequency-domain representation and/or a spatial-domain representationto facilitate and/or support detecting and/or identifying one or more ofgestures 502 and/or motion 504.

In some examples, wearable 102 and/or head-mounted display 104 mayresolve and/or decompose such EMG signals as represented in the timedomain. For example, wearable 102 and/or head-mounted display 104 mayperform a decomposition 614 of EMG signal 612 into motor unit actionpotential trains 616 representative of individual motor neurons 610 thatinnervate muscles 604 in the user’s body. Wearable 102 and/orhead-mounted display 104 may determine that the user performed one ormore of gestures 502 and/or motion 504 based at least in part on certainpatterns and/or spikes identified and/or detected in motor unit actionpotential trains 616.

FIG. 7 illustrates exemplary representations of neuromuscular signals700 detected and/or identified via wearable 102 donned by a useroperating artificial-reality system 100. In some examples, neuromuscularsignals 700 may include and/or represent raw time-domain EMG signals 702detected via electrodes 116 arranged and/or disposed on wearable 102. Inone example, wearable 102 and/or head-mounted display 104 may process,translate, and/or convert raw time-domain EMG signals 702 into processedfrequency-domain EMG signals 704. In this example, wearable 102 and/orhead-mounted display 104 may determine that the user performed one ormore of gestures 502 and/or motion 504 based at least in part on certainpatterns and/or spikes identified and/or detected in raw time-domain EMGsignals 702 and/or processed frequency-domain EMG signals 704.

FIG. 9 illustrates an exemplary point-of-view implementation 900 ofartificial-reality system 100. As illustrated in FIG. 9 , exemplarypoint-of-view implementation 900 may include and/or represent a userdonning head-mounted display 104 and wearable 102. In some examples,display screen 110 of head-mounted display 104 may include and/orrepresent lenses that facilitate and/or support see-through visibilitywith superimposed virtual overlays. For example, wearable 102 maytransmit precise time-stamped UWB-IR signals to various UWB antennasincorporated into head-mounted display 104. In this example,head-mounted display 104 may be able to determine the angle of arrivalfor the time-stamped signals. Based on the angle of arrival,head-mounted display 104 may be able to triangulate and/or track therelative location of a hand 902 proximate to wearable 102. Head-mounteddisplay 104 may then display a virtual component 904 that corresponds tothe location of hand 902 in the field of view of display screen 110.

In some examples, head-mounted display 104 may superimpose a virtualcomponent 906 over certain real components visible via display screen110. For example, virtual component 904 may include and/or represent apointer controlled by movements and/or gestures of wearable 102 and/orhand 902, and virtual component 906 may include and/or represent amessage, dialog box, and/or modal window. In one example, the user mayperform one or more gestures and/or movements in connection with virtualcomponent 906 to interact with, open, control, modify, and/or otherwisemanipulate virtual component 906. In this example, wearable 102 maydetect neuromuscular signals indicative of such gestures and/ormovements, and head-mounted display 104 may receive data indicative ofsuch gestures and/or movements from wearable 102. Head-mounted display104 may then determine that the user performed such gestures and/ormovements as virtual component 904 overlapped with virtual component906. As a result, head-mounted display 104 may perform one or moreactions (e.g., clicking, rotating, dragging, dropping, zooming, etc.)mapped to such gestures and/or movements in connection with virtualcomponent 906.

FIG. 10 illustrates an exemplary spherical coordinate system 1000 fortranslating user input to display screen 110 of head-mounted display104. As illustrated in FIG. 10 , exemplary spherical coordinate system1000 may include and/or represent an azimuth 1010 and an elevation 1020.In some examples, head-mounted display 104 may present a virtualcomponent (e.g., a pointer and/or cursor) at a location and/or positiondefined by azimuth 1010 and elevation 1020 of spherical coordinatesystem 1000 within the field of view of display screen 110.

FIG. 11 is a flow diagram of an exemplary method 1100 for detecting userinput via hand gestures and arm movements. In one example, the stepsshown in FIG. 11 may be performed during operation of anartificial-reality system. Additionally or alternatively, the stepsshown in FIG. 11 may also incorporate and/or involve various sub-stepsand/or variations consistent with the descriptions provided above inconnection with FIGS. 1-10 .

As illustrated in FIG. 11 , method 1100 may include and/or involve thestep of detecting, by a wearable, one or more neuromuscular signals at abody part of a user (1110). Step 1110 may be performed in a variety ofways, including any of those described above in connection with FIGS.1-10 . For example, a wearable donned by a user of an artificial-realitysystem may detect one or more neuromuscular signals at a body part of auser.

Method 1100 may also include and/or involve the step of emitting anelectromagnetic signal by a transmitter included on the wearable (1120).Step 1120 may be performed in a variety of ways, including any of thosedescribed above in connection with FIGS. 1-10 . For example, thewearable may include a transmitter that emits and/or transmits a UWB-IRsignal to an array of UWB antennas arranged on the head-mounted display.

Method 1100 may further include and/or involve the step of receiving theelectromagnetic signal emitted by the transmitter at a set of receiversincluded on the head-mounted display (1130). Step 1130 may be performedin a variety of ways, including any of those described above inconnection with FIGS. 1-10 . For example, a set of UWB antennas arrangedon the head-mounted display may detect and/or receive the UWB-IR signalemitted and/or transmitted by the transmitter included on the wearable.

Method 1100 may additionally include and/or involve the step ofdetermining that the user has made a specific gesture based at least inpart on the neuromuscular signals (1140). Step 1140 may be performed ina variety of ways, including any of those described above in connectionwith FIGS. 1-10 . For example, a processing device incorporated in thewearable or the head-mounted display may determine and/or identify aspecific gesture and/or movement made by the user based at least in parton the neuromuscular signals.

Method 1100 may also include and/or involve the step of determining aposition of the body part of the user when the user made the specificgesture based at least in part on the electromagnetic signal (1150).Step 1150 may be performed in a variety of ways, including any of thosedescribed above in connection with FIGS. 1-10 . For example, aprocessing device incorporated in the head-mounted display may determineand/or identify a position of the body part of the user when the usermade the specific gesture based at least in part on the electromagneticsignal received from the wearable.

As described above in connection with FIGS. 1-11 , AR glasses and/orcorresponding systems may facilitate and/or support multidimensionalvirtual cursor movement and object selection through arm movement andhand gestures. In some examples, AR glasses may incorporate a set of UWBantennas that receive various information and/or data from a wristbandworn by a user. In one example, the wristband may include a UWB impulseradio that transmits precise time-stamped signals to the set of UWBantennas incorporated into the AR glasses. In this example, the ARglasses may be able to determine the angle of arrival for thetime-stamped signals. Based on the angle of arrival, the AR glasses maybe able to triangulate and/or track the relative location of the user’shand proximate to the wristband. The AR glasses may then display avirtual representation of the user’s hand and/or a cursor in the user’sfield of view.

In addition, the wristband may include a set of EMG sensors that measureEMG activity at the user’s wrist. In this example, the wristband and/orthe AR glasses may be able to decipher certain hand gestures performedby the user while operating the AR glasses based at least in part on theEMG activity measured by the EMG sensors. The wristband and/or the ARglasses may then determine whether any of those hand gestures correspondto certain commands and/or user input to be applied to the user’s ARexperience. For example, the user may point to real and/or virtualcomponents displayed in the user’s AR experience, and the AR glasses maydetermine which real and/or virtual components the user is pointing tobased at least in part on the angle of arrival of the UWB-transmittedsignals. The user may then select such components in the AR experienceby performing a certain gesture with his or her hands, as detected byEMG activity measured by the EMG sensors.

Example Embodiments

Example 1: An artificial-reality system comprising (1) a wearabledimensioned to be donned on a body part of a user, wherein the wearablecomprises (A) a set of electrodes that detect one or more neuromuscularsignals via the body part of the user and (B) a transmitter thattransmits an electromagnetic signal, (2) a head-mounted displaycommunicatively coupled to the wearable, wherein the head-mounteddisplay comprises a set of receivers that receive the electromagneticsignal transmitted by the transmitter included on the wearable, and (3)one or more processing devices that (1) determine, based at least inpart on the neuromuscular signals detected via the body part of theuser, that the user has made a specific gesture and (2) determine, basedat least in part on the electromagnetic signal received by the set ofreceivers included on the head-mounted display, a position of the bodypart of the user when the user made the specific gesture.

Example 2: The artificial-reality system of Example 1, wherein at leastone of the processing devices is incorporated in the wearable.

Example 3: The artificial-reality system of Example 1 or 2, wherein atleast one of the processing devices is incorporated in the head-mounteddisplay.

Example 4: The artificial-reality system of any of Examples 1-3, wherein(1) the wearable comprises a first Bluetooth radio and (2) thehead-mounted display comprises a second Bluetooth radio that iscommunicatively coupled to the first Bluetooth radio, the first andsecond Bluetooth radios being configured to exchange configuration databetween the wearable and the head-mounted display.

Example 5: The artificial-reality system of any of Examples 1-4, whereinthe first and second Bluetooth radios are further configured to exchangedata about the neuromuscular signals between the wearable and thehead-mounted display.

Example 6: The artificial-reality system of any of Examples 1-5, whereinat least one of the processing devices generates, based at least in parton the data about the neuromuscular signals, an input command thatcauses the head-mounted display to modify at least one virtual componentto account for the specific gesture.

Example 7: The artificial-reality system of any of Examples 1-6, whereinthe transmitter incorporates a time stamp into the electromagneticsignal before transmitting the electromagnetic signal to the set ofreceivers.

Example 8: The artificial-reality system of any of Examples 1-7, whereinat least one of the processing devices (1) determines a first time ofarrival for the electromagnetic signal as received by a first receiverincluded in the set of receivers, (2) determines a second time ofarrival for the electromagnetic signal as received by a second receiverincluded in the set of receivers, and (3) calculates, based at least inpart on the first and second times of arrival for the electromagneticsignal and the time stamp, an angle of arrival for the electromagneticsignal relative to the set of receivers.

Example 9: The artificial-reality system of any of Examples 1-8, whereinat least one of the processing devices (1) calculates, based at least inpart on the angle of arrival, at least one dimension for a position of avirtual component within a field of view of the head-mounted display and(2) presents the virtual component at the position within the field ofview of the head-mounted display based at least in part on thedimension.

Example 10: The artificial-reality system of any of Examples 1-9,wherein the dimension calculated for the position of the virtualcomponent comprises at least one of (1) an azimuth for the virtualcomponent to be presented within the field of view of the head-mounteddisplay, (2) an elevation for the virtual component to be presentedwithin the field of view of the head-mounted display, or (3) a depth forthe virtual component to be presented within the field of view of thehead-mounted display.

Example 11: The artificial-reality system of any of Examples 1-10,wherein at least one of the processing devices (1) determines a firstphase of the electromagnetic signal as received by the first receiverincluded in the set of receivers, (2) determines a second phase of theelectromagnetic signal as received by the second receiver included inthe set of receivers, and (3) calculates, based at least in part on adifference between the first and second phases of the electromagneticsignal and the time stamp, an angle of arrival for the electromagneticsignal relative to the set of receivers.

Example 12: The artificial-reality system of any of Examples 1-11,wherein at least one of the processing devices (1) calculates, based atleast in part on the angle of arrival, a two-dimensional position forthe virtual component within the field of view of the head-mounteddisplay and (2) presents the virtual component at the two-dimensionalposition within the field of view of the head-mounted display.

Example 13: The artificial-reality system of any of Examples 1-12,wherein at least one of the processing devices (1) calculates, based atleast in part on the angle of arrival, a three-dimensional position forthe virtual component within the field of view of the head-mounteddisplay and (2) presents the virtual component at the three-dimensionalposition within the field of view of the head-mounted display.

Example 14: The artificial-reality system of any of Examples 1-13,wherein (1) the virtual component presented at the position comprises apointer presented at the position and (2) at least one of the processingdevices superimposes the pointer over a screen of the head-mounteddisplay.

Example 15: The artificial-reality system of any of Examples 1-14,wherein at least one of the processing devices generates, based at leastin part on data about the neuromuscular signals, an input command thatcauses the head-mounted display to modify at least one additionalvirtual component presented proximate to the pointer within a field ofview of the head-mounted display to account for the specific gesture.

Example 16: The artificial-reality system of any of Examples 1-15,wherein at least one of the processing devices (1) determines, based atleast in part on the angle of arrival, that the wearable is no longervisible within the field of view of the head-mounted display and, inresponse to determining that the wearable is no longer visible withinthe field of view of the head-mounted display, (2) removing the virtualcomponent from the field of view of the head-mounted display.

Example 17: The artificial-reality system of any of Examples 1-16,further comprising an additional wearable dimensioned to be donned on anadditional body part of the user, wherein the wearable comprises (1) anadditional set of electrodes that detect one or more additionalneuromuscular signals via the additional body part of the user and (2)at least one additional transmitter that transmits an additionalelectromagnetic signal, wherein the head-mounted display is alsocommunicatively coupled to the additional wearable, the set of receiversreceiving the additional electromagnetic signal transmitted by theadditional transmitter included on the additional wearable and at leastone of the processing devices (1) determines, based at least in part onthe additional neuromuscular signals detected via the body part of theuser, that the user has made an additional gesture and (2) determines,based at least in part on the additional electromagnetic signal receivedby the set of receivers included on the head-mounted display, a positionof the body part of the user when the user made the additional gesture.

Example 18: The artificial-reality system of any of Examples 1-17,wherein at least one of the processing devices (1) calculates, based atleast in part on the electromagnetic signal, at least one dimension fora position of a virtual component within a field of view of thehead-mounted display, (2) calculates, based at least in part on theadditional electromagnetic signal, at least one additional dimension foran additional position of an additional virtual component within thefield of view of the head-mounted display, and (3) simultaneouslypresents, within the field of view of the head-mounted display, thevirtual component at the position and the additional virtual componentat the additional position based at least in part on the dimension andthe additional dimension.

Example 19: A head-mounted display comprising (1) a set of receiversconfigured to receive an electromagnetic signal transmitted by atransmitter included on a wearable dimensioned to be donned on a bodypart of a user, (2) a radio configured to receive data about one or moreneuromuscular signals detected by the wearable via the body part of theuser, and (3) at least one processing device communicatively coupled tothe set of receivers and the radio, wherein the processing device (A)determines, based at least in part on the data about the neuromuscularsignals detected via the body part of the user, that the user has made aspecific gesture and (B) determines, based at least in part on theelectromagnetic signal received by the set of receivers included on thehead-mounted display, a position of the body part of the user when theuser made the specific gesture.

Example 20: A method comprising (1) detecting, by a wearable donned on abody part of a user, one or more neuromuscular signals at the body partof the user, (2) emitting, by a transmitter included on the wearable, anelectromagnetic signal, (3) receiving, by a set of receivers included ona head-mounted display donned by the user, the electromagnetic signalemitted by the transmitter included on the wearable, (4) determining, byone or more processing devices, that the user has made a specificgesture based at least in part on the neuromuscular signals, and (5)determining, by the one or more processing devices, a position of thebody part of the user when the user made the specific gesture based atleast in part on the electromagnetic signal.

Embodiments of the present disclosure may include or be implemented inconjunction with various types of artificial-reality systems. Artificialreality is a form of reality that has been adjusted in some mannerbefore presentation to a user, which may include, for example, a virtualreality, an augmented reality, a mixed reality, a hybrid reality, orsome combination and/or derivative thereof. Artificial-reality contentmay include completely computer-generated content or computer-generatedcontent combined with captured (e.g., real-world) content. Theartificial-reality content may include video, audio, haptic feedback, orsome combination thereof, any of which may be presented in a singlechannel or in multiple channels (such as stereo video that produces a 3Deffect to the viewer). Additionally, in some embodiments, artificialreality may also be associated with applications, products, accessories,services, or some combination thereof, that are used to, for example,create content in an artificial reality and/or are otherwise used in(e.g., to perform activities in) an artificial reality.

Artificial-reality systems may be implemented in a variety of differentform factors and configurations. Some artificial-reality systems may bedesigned to work without near-eye displays (NEDs). Otherartificial-reality systems may include an NED that also providesvisibility into the real world (such as, e.g., augmented-reality system1200 in FIG. 12 ) or that visually immerses a user in an artificialreality (such as, e.g., virtual-reality system 1300 in FIG. 13 ). Whilesome artificial-reality devices may be self-contained systems, otherartificial-reality devices may communicate and/or coordinate withexternal devices to provide an artificial-reality experience to a user.Examples of such external devices include handheld controllers, mobiledevices, desktop computers, devices worn by a user, devices worn by oneor more other users, and/or any other suitable external system.

Turning to FIG. 12 , augmented-reality system 1200 may include aneyewear device 1202 with a frame 1210 configured to hold a left displaydevice 1215(A) and a right display device 1215(B) in front of a user’seyes. Display devices 1215(A) and 1215(B) may act together orindependently to present an image or series of images to a user. Whileaugmented-reality system 1200 includes two displays, embodiments of thisdisclosure may be implemented in augmented-reality systems with a singleNED or more than two NEDs.

In some embodiments, augmented-reality system 1200 may include one ormore sensors, such as sensor 1240. Sensor 1240 may generate measurementsignals in response to motion of augmented-reality system 1200 and maybe located on substantially any portion of frame 1210. Sensor 1240 mayrepresent one or more of a variety of different sensing mechanisms, suchas a position sensor, an inertial measurement unit (IMU), a depth cameraassembly, a structured light emitter and/or detector, or any combinationthereof. In some embodiments, augmented-reality system 1200 may or maynot include sensor 1240 or may include more than one sensor. Inembodiments in which sensor 1240 includes an IMU, the IMU may generatecalibration data based on measurement signals from sensor 1240. Examplesof sensor 1240 may include, without limitation, accelerometers,gyroscopes, magnetometers, other suitable types of sensors that detectmotion, sensors used for error correction of the IMU, or somecombination thereof.

In some examples, augmented-reality system 1200 may also include amicrophone array with a plurality of acoustic transducers1220(A)-1220(J), referred to collectively as acoustic transducers 1220.Acoustic transducers 1220 may represent transducers that detect airpressure variations induced by sound waves. Each acoustic transducer1220 may be configured to detect sound and convert the detected soundinto an electronic format (e.g., an analog or digital format). Themicrophone array in FIG. 12 may include, for example, ten acoustictransducers: 1220(A) and 1220(B), which may be designed to be placedinside a corresponding ear of the user, acoustic transducers 1220(C),1220(D), 1220(E), 1220(F), 1220(G), and 1220(H), which may be positionedat various locations on frame 1210, and/or acoustic transducers 1220(I)and 1220(J), which may be positioned on a corresponding neckband 1205.

In some embodiments, one or more of acoustic transducers 1220(A)-(J) maybe used as output transducers (e.g., speakers). For example, acoustictransducers 1220(A) and/or 1220(B) may be earbuds or any other suitabletype of headphone or speaker.

The configuration of acoustic transducers 1220 of the microphone arraymay vary. While augmented-reality system 1200 is shown in FIG. 12 ashaving ten acoustic transducers 1220, the number of acoustic transducers1220 may be greater or less than ten. In some embodiments, using highernumbers of acoustic transducers 1220 may increase the amount of audioinformation collected and/or the sensitivity and accuracy of the audioinformation. In contrast, using a lower number of acoustic transducers1220 may decrease the computing power required by an associatedcontroller 1250 to process the collected audio information. In addition,the position of each acoustic transducer 1220 of the microphone arraymay vary. For example, the position of an acoustic transducer 1220 mayinclude a defined position on the user, a defined coordinate on frame1210, an orientation associated with each acoustic transducer 1220, orsome combination thereof.

Acoustic transducers 1220(A) and 1220(B) may be positioned on differentparts of the user’s ear, such as behind the pinna, behind the tragus,and/or within the auricle or fossa. Or, there may be additional acoustictransducers 1220 on or surrounding the ear in addition to acoustictransducers 1220 inside the ear canal. Having an acoustic transducer1220 positioned next to an ear canal of a user may enable the microphonearray to collect information on how sounds arrive at the ear canal. Bypositioning at least two of acoustic transducers 1220 on either side ofa user’s head (e.g., as binaural microphones), augmented-reality system1200 may simulate binaural hearing and capture a 3D stereo sound fieldaround a user’s head. In some embodiments, acoustic transducers 1220(A)and 1220(B) may be connected to augmented-reality system 1200 via awired connection 1230, and in other embodiments acoustic transducers1220(A) and 1220(B) may be connected to augmented-reality system 1200via a wireless connection (e.g., a BLUETOOTH connection). In still otherembodiments, acoustic transducers 1220(A) and 1220(B) may not be used atall in conjunction with augmented-reality system 1200.

Acoustic transducers 1220 on frame 1210 may be positioned in a varietyof different ways, including along the length of the temples, across thebridge, above or below display devices 1215(A) and 1215(B), or somecombination thereof. Acoustic transducers 1220 may also be oriented suchthat the microphone array is able to detect sounds in a wide range ofdirections surrounding the user wearing the augmented-reality system1200. In some embodiments, an optimization process may be performedduring manufacturing of augmented-reality system 1200 to determinerelative positioning of each acoustic transducer 1220 in the microphonearray.

In some examples, augmented-reality system 1200 may include or beconnected to an external device (e.g., a paired device), such asneckband 1205. Neckband 1205 generally represents any type or form ofpaired device. Thus, the following discussion of neckband 1205 may alsoapply to various other paired devices, such as charging cases, smartwatches, smart phones, wrist bands, other wearable devices, hand-heldcontrollers, tablet computers, laptop computers, other external computedevices, etc.

As shown, neckband 1205 may be coupled to eyewear device 1202 via one ormore connectors. The connectors may be wired or wireless and may includeelectrical and/or non-electrical (e.g., structural) components. In somecases, eyewear device 1202 and neckband 1205 may operate independentlywithout any wired or wireless connection between them. While FIG. 12illustrates the components of eyewear device 1202 and neckband 1205 inexample locations on eyewear device 1202 and neckband 1205, thecomponents may be located elsewhere and/or distributed differently oneyewear device 1202 and/or neckband 1205. In some embodiments, thecomponents of eyewear device 1202 and neckband 1205 may be located onone or more additional peripheral devices paired with eyewear device1202, neckband 1205, or some combination thereof.

Pairing external devices, such as neckband 1205, with augmented-realityeyewear devices may enable the eyewear devices to achieve the formfactor of a pair of glasses while still providing sufficient battery andcomputation power for expanded capabilities. Some or all of the batterypower, computational resources, and/or additional features ofaugmented-reality system 1200 may be provided by a paired device orshared between a paired device and an eyewear device, thus reducing theweight, heat profile, and form factor of the eyewear device overallwhile still retaining desired functionality. For example, neckband 1205may allow components that would otherwise be included on an eyeweardevice to be included in neckband 1205 since users may tolerate aheavier weight load on their shoulders than they would tolerate on theirheads. Neckband 1205 may also have a larger surface area over which todiffuse and disperse heat to the ambient environment. Thus, neckband1205 may allow for greater battery and computation capacity than mightotherwise have been possible on a stand-alone eyewear device. Sinceweight carried in neckband 1205 may be less invasive to a user thanweight carried in eyewear device 1202, a user may tolerate wearing alighter eyewear device and carrying or wearing the paired device forgreater lengths of time than a user would tolerate wearing a heavystandalone eyewear device, thereby enabling users to more fullyincorporate artificial-reality environments into their day-to-dayactivities.

Neckband 1205 may be communicatively coupled with eyewear device 1202and/or to other devices. These other devices may provide certainfunctions (e.g., tracking, localizing, depth mapping, processing,storage, etc.) to augmented-reality system 1200. In the embodiment ofFIG. 12 , neckband 1205 may include two acoustic transducers (e.g.,1220(I) and 1220(J)) that are part of the microphone array (orpotentially form their own microphone subarray). Neckband 1205 may alsoinclude a controller 1225 and a power source 1235.

Acoustic transducers 1220(I) and 1220(J) of neckband 1205 may beconfigured to detect sound and convert the detected sound into anelectronic format (analog or digital). In the embodiment of FIG. 12 ,acoustic transducers 1220(I) and 1220(J) may be positioned on neckband1205, thereby increasing the distance between the neckband acoustictransducers 1220(I) and 1220(J) and other acoustic transducers 1220positioned on eyewear device 1202. In some cases, increasing thedistance between acoustic transducers 1220 of the microphone array mayimprove the accuracy of beamforming performed via the microphone array.For example, if a sound is detected by acoustic transducers 1220(C) and1220(D) and the distance between acoustic transducers 1220(C) and1220(D) is greater than, e.g., the distance between acoustic transducers1220(D) and 1220(E), the determined source location of the detectedsound may be more accurate than if the sound had been detected byacoustic transducers 1220(D) and 1220(E).

Controller 1225 of neckband 1205 may process information generated bythe sensors on neckband 1205 and/or augmented-reality system 1200. Forexample, controller 1225 may process information from the microphonearray that describes sounds detected by the microphone array. For eachdetected sound, controller 1225 may perform a direction-of-arrival (DOA)estimation to estimate a direction from which the detected sound arrivedat the microphone array. As the microphone array detects sounds,controller 1225 may populate an audio data set with the information. Inembodiments in which augmented-reality system 1200 includes an inertialmeasurement unit, controller 1225 may compute all inertial and spatialcalculations from the IMU located on eyewear device 1202. A connectormay convey information between augmented-reality system 1200 andneckband 1205 and between augmented-reality system 1200 and controller1225. The information may be in the form of optical data, electricaldata, wireless data, or any other transmittable data form. Moving theprocessing of information generated by augmented-reality system 1200 toneckband 1205 may reduce weight and heat in eyewear device 1202, makingit more comfortable to the user.

Power source 1235 in neckband 1205 may provide power to eyewear device1202 and/or to neckband 1205. Power source 1235 may include, withoutlimitation, lithium-ion batteries, lithium-polymer batteries, primarylithium batteries, alkaline batteries, or any other form of powerstorage. In some cases, power source 1235 may be a wired power source.Including power source 1235 on neckband 1205 instead of on eyeweardevice 1202 may help better distribute the weight and heat generated bypower source 1235.

As noted, some artificial-reality systems may, instead of blending anartificial reality with actual reality, substantially replace one ormore of a user’s sensory perceptions of the real world with a virtualexperience. One example of this type of system is a head-worn displaysystem, such as virtual-reality system 1300 in FIG. 13 , that mostly orcompletely covers a user’s field of view. Virtual-reality system 1300may include a front rigid body 1302 and a band 1304shaped to fit arounda user’s head. Virtual-reality system 1300 may also include output audiotransducers 1306(A) and 1306(B). Furthermore, while not shown in FIG. 13, front rigid body 1302 may include one or more electronic elements,including one or more electronic displays, one or more inertialmeasurement units (IMUs), one or more tracking emitters or detectors,and/or any other suitable device or system for creating anartificial-reality experience.

Artificial-reality systems may include a variety of types of visualfeedback mechanisms. For example, display devices in augmented-realitysystem 1200 and/or virtual-reality system 1300 may include one or moreliquid crystal displays (LCDs), light-emitting diode (LED) displays,microLED displays, organic LED (OLED) displays, digital light project(DLP) micro-displays, liquid crystal on silicon (LCoS) micro-displays,and/or any other suitable type of display screen. Theseartificial-reality systems may include a single display screen for botheyes or may provide a display screen for each eye, which may allow foradditional flexibility for varifocal adjustments or for correcting auser’s refractive error. Some of these artificial-reality systems mayalso include optical subsystems having one or more lenses (e.g., concaveor convex lenses, Fresnel lenses, adjustable liquid lenses, etc.)through which a user may view a display screen. These optical subsystemsmay serve a variety of purposes, including to collimate (e.g., make anobject appear at a greater distance than its physical distance), tomagnify (e.g., make an object appear larger than its actual size),and/or to relay (to, e.g., the viewer’s eyes) light. These opticalsubsystems may be used in a non-pupil-forming architecture (such as asingle lens configuration that directly collimates light but results inso-called pincushion distortion) and/or a pupil-forming architecture(such as a multi-lens configuration that produces so-called barreldistortion to nullify pincushion distortion).

In addition to or instead of using display screens, some of theartificial-reality systems described herein may include one or moreprojection systems. For example, display devices in augmented-realitysystem 1200 and/or virtual-reality system 1300 may include microLEDprojectors that project light (using, e.g., a waveguide) into displaydevices, such as clear combiner lenses that allow ambient light to passthrough. The display devices may refract the projected light toward auser’s pupil and may enable a user to simultaneously view bothartificial-reality content and the real world. The display devices mayaccomplish this using any of a variety of different optical components,including waveguide components (e.g., holographic, planar, diffractive,polarized, and/or reflective waveguide elements), light-manipulationsurfaces and elements (such as diffractive, reflective, and refractiveelements and gratings), coupling elements, etc. Artificial-realitysystems may also be configured with any other suitable type or form ofimage projection system, such as retinal projectors used in virtualretina displays.

The artificial-reality systems described herein may also include varioustypes of computer vision components and subsystems. For example,augmented-reality system 1200 and/or virtual-reality system 1300 mayinclude one or more optical sensors, such as 2D or 3D cameras,structured light transmitters and detectors, time-of-flight depthsensors, single-beam or sweeping laser rangefinders, 3D LiDAR sensors,and/or any other suitable type or form of optical sensor. Anartificial-reality system may process data from one or more of thesesensors to identify a location of a user, to map the real world, toprovide a user with context about real-world surroundings, and/or toperform a variety of other functions.

The artificial-reality systems described herein may also include one ormore input and/or output audio transducers. Output audio transducers mayinclude voice coil speakers, ribbon speakers, electrostatic speakers,piezoelectric speakers, bone conduction transducers, cartilageconduction transducers, tragus-vibration transducers, and/or any othersuitable type or form of audio transducer. Similarly, input audiotransducers may include condenser microphones, dynamic microphones,ribbon microphones, and/or any other type or form of input transducer.In some embodiments, a single transducer may be used for both audioinput and audio output.

In some embodiments, the artificial-reality systems described herein mayalso include tactile (i.e., haptic) feedback systems, which may beincorporated into headwear, gloves,bodysuits, handheld controllers,environmental devices (e.g., chairs, floor mats, etc.), and/or any othertype of device or system. Haptic feedback systems may provide varioustypes of cutaneous feedback, including vibration, force, traction,texture, and/or temperature. Haptic feedback systems may also providevarious types of kinesthetic feedback, such as motion and compliance.Haptic feedback may be implemented using motors, piezoelectricactuators, fluidic systems, and/or a variety of other types of feedbackmechanisms. Haptic feedback systems may be implemented independent ofother artificial-reality devices, within other artificial-realitydevices, and/or in conjunction with other artificial-reality devices.

By providing haptic sensations, audible content, and/or visual content,artificial-reality systems may create an entire virtual experience orenhance a user’s real-world experience in a variety of contexts andenvironments. For instance, artificial-reality systems may assist orextend a user’s perception, memory, or cognition within a particularenvironment. Some systems may enhance a user’s interactions with otherpeople in the real world or may enable more immersive interactions withother people in a virtual world. Artificial-reality systems may also beused for educational purposes (e.g., for teaching or training inschools, hospitals, government organizations, military organizations,business enterprises, etc.), entertainment purposes (e.g., for playingvideo games, listening to music, watching video content, etc.), and/orfor accessibility purposes (e.g., as hearing aids, visual aids, etc.).The embodiments disclosed herein may enable or enhance a user’sartificial-reality experience in one or more of these contexts andenvironments and/or in other contexts and environments.

Some augmented-reality systems may map a user’s and/or device’senvironment using techniques referred to as “simultaneous location andmapping” (SLAM). SLAM mapping and location identifying techniques mayinvolve a variety of hardware and software tools that can create orupdate a map of an environment while simultaneously keeping track of auser’s location within the mapped environment. SLAM may use manydifferent types of sensors to create a map and determine a user’sposition within the map.

SLAM techniques may, for example, implement optical sensors to determinea user’s location. Radios including WiFi, BLUETOOTH, global positioningsystem (GPS), cellular or other communication devices may be also usedto determine a user’s location relative to a radio transceiver or groupof transceivers (e.g., a WiFi router or group of GPS satellites).Acoustic sensors such as microphone arrays or 2D or 3D sonar sensors mayalso be used to determine a user’s location within an environment.Augmented-reality and virtual-reality devices (such as systems 1200 and1300 of FIGS. 12 and 13 , respectively) may incorporate any or all ofthese types of sensors to perform SLAM operations such as creating andcontinually updating maps of the user’s current environment. In at leastsome of the embodiments described herein, SLAM data generated by thesesensors may be referred to as “environmental data” and may indicate auser’s current environment. This data may be stored in a local or remotedata store (e.g., a cloud data store) and may be provided to a user’sAR/VR device on demand.

As noted, artificial-reality systems 1200 and 1300 may be used with avariety of other types of devices to provide a more compellingartificial-reality experience. These devices may be haptic interfaceswith transducers that provide haptic feedback and/or that collect hapticinformation about a user’s interaction with an environment. Theartificial-reality systems disclosed herein may include various types ofhaptic interfaces that detect or convey various types of hapticinformation, including tactile feedback (e.g., feedback that a userdetects via nerves in the skin, which may also be referred to ascutaneous feedback) and/or kinesthetic feedback (e.g., feedback that auser detects via receptors located in muscles, joints, and/or tendons).

Haptic feedback may be provided by interfaces positioned within a user’senvironment (e.g., chairs, tables, floors, etc.) and/or interfaces onarticles that may be worn or carried by a user (e.g., gloves,wristbands, etc.). As an example, FIG. 14 illustrates a vibrotactilesystem 1400 in the form of a wearable glove (haptic device 1410) andwristband (haptic device 1420). Haptic device 1410 and haptic device1420 are shown as examples of wearable devices that include a flexible,wearable textile material 1430 that is shaped and configured forpositioning against a user’s hand and wrist, respectively. Thisdisclosure also includes vibrotactile systems that may be shaped andconfigured for positioning against other human body parts, such as afinger, an arm, a head, a torso, a foot, or a leg. By way of example andnot limitation, vibrotactile systems according to various embodiments ofthe present disclosure may also be in the form of a glove, a headband,an armband, a sleeve, a head covering, a sock, a shirt, or pants, amongother possibilities. In some examples, the term “textile” may includeany flexible, wearable material, including woven fabric, non-wovenfabric, leather, cloth, a flexible polymer material, compositematerials, etc.

One or more vibrotactile devices 1440 may be positioned at leastpartially within one or more corresponding pockets formed in textilematerial 1430 of vibrotactile system 1400. Vibrotactile devices 1440 maybe positioned in locations to provide a vibrating sensation (e.g.,haptic feedback) to a user of vibrotactile system 1400. For example,vibrotactile devices 1440 may be positioned against the user’sfinger(s), thumb, or wrist, as shown in FIG. 14 . Vibrotactile devices1440 may, in some examples, be sufficiently flexible to conform to orbend with the user’s corresponding body part(s).

A power source 1450 (e.g., a battery) for applying a voltage to thevibrotactile devices 1440 for activation thereof may be electricallycoupled to vibrotactile devices 1440, such as via conductive wiring1452. In some examples, each of vibrotactile devices 1440 may beindependently electrically coupled to power source 1450 for individualactivation. In some embodiments, a processor 1460 may be operativelycoupled to power source 1450 and configured (e.g., programmed) tocontrol activation of vibrotactile devices 1440.

Vibrotactile system 1400 may be implemented in a variety of ways. Insome examples, vibrotactile system 1400 may be a standalone system withintegral subsystems and components for operation independent of otherdevices and systems. As another example, vibrotactile system 1400 may beconfigured for interaction with another device or system 1470. Forexample, vibrotactile system 1400 may, in some examples, include acommunications interface 1480 for receiving and/or sending signals tothe other device or system 1470. The other device or system 1470 may bea mobile device, a gaming console, an artificial-reality (e.g.,virtual-reality, augmented-reality, mixed-reality) device, a personalcomputer, a tablet computer, a network device (e.g., a modem, a router,etc.), a handheld controller, etc. Communications interface 1480 mayenable communications between vibrotactile system 1400 and the otherdevice or system 1470 via a wireless (e.g., Wi-Fi, BLUETOOTH, cellular,radio, etc.) link or a wired link. If present, communications interface1480 may be in communication with processor 1460, such as to provide asignal to processor 1460 to activate or deactivate one or more of thevibrotactile devices 1440.

Vibrotactile system 1400 may optionally include other subsystems andcomponents, such as touch-sensitive pads 1490, pressure sensors, motionsensors, position sensors, lighting elements, and/or user interfaceelements (e.g., an on/off button, a vibration control element, etc.).During use, vibrotactile devices 1440 may be configured to be activatedfor a variety of different reasons, such as in response to the user’sinteraction with user interface elements, a signal from the motion orposition sensors, a signal from the touch-sensitive pads 1490, a signalfrom the pressure sensors, a signal from the other device or system1470, etc.

Although power source 1450, processor 1460, and communications interface1480 are illustrated in FIG. 14 as being positioned in haptic device1420, the present disclosure is not so limited. For example, one or moreof power source 1450, processor 1460, or communications interface 1480may be positioned within haptic device 1410 or within another wearabletextile.

Haptic wearables, such as those shown in and described in connectionwith FIG. 14 , may be implemented in a variety of types ofartificial-reality systems and environments. FIG. 15 shows an exampleartificial-reality environment 1500 including one head-mountedvirtual-reality display and two haptic devices (i.e., gloves), and inother embodiments any number and/or combination of these components andother components may be included in an artificial-reality system. Forexample, in some embodiments there may be multiple head-mounted displayseach having an associated haptic device, with each head-mounted displayand each haptic device communicating with the same console, portablecomputing device, or other computing system.

Head-mounted display 1502 generally represents any type or form ofvirtual-reality system, such as virtual-reality system 1300 in FIG. 13 .Haptic device 1504 generally represents any type or form of wearabledevice, worn by a user of an artificial-reality system, that provideshaptic feedback to the user to give the user the perception that he orshe is physically engaging with a virtual object. In some embodiments,haptic device 1504 may provide haptic feedback by applying vibration,motion, and/or force to the user. For example, haptic device 1504 maylimit or augment a user’s movement. To give a specific example, hapticdevice 1504 may limit a user’s hand from moving forward so that the userhas the perception that his or her hand has come in physical contactwith a virtual wall. In this specific example, one or more actuatorswithin the haptic device may achieve the physical-movement restrictionby pumping fluid into an inflatable bladder of the haptic device. Insome examples, a user may also use haptic device 1504 to send actionrequests to a console. Examples of action requests include, withoutlimitation, requests to start an application and/or end the applicationand/or requests to perform a particular action within the application.

While haptic interfaces may be used with virtual-reality systems, asshown in FIG. 15 , haptic interfaces may also be used withaugmented-reality systems, as shown in FIG. 16 . FIG. 16 is aperspective view of a user 1610 interacting with an augmented-realitysystem 1600. In this example, user 1610 may wear a pair ofaugmented-reality glasses 1620 that may have one or more displays 1622and that are paired with a haptic device 1630. In this example, hapticdevice 1630 may be a wristband that includes a plurality of bandelements 1632 and a tensioning mechanism 1634 that connects bandelements 1632 to one another.

One or more of band elements 1632 may include any type or form ofactuator suitable for providing haptic feedback. For example, one ormore of band elements 1632 may be configured to provide one or more ofvarious types of cutaneous feedback, including vibration, force,traction, texture, and/or temperature. To provide such feedback, bandelements 1632 may include one or more of various types of actuators. Inone example, each of band elements 1632 may include a vibrotactor (e.g.,a vibrotactile actuator) configured to vibrate in unison orindependently to provide one or more of various types of hapticsensations to a user. Alternatively, only a single band element or asubset of band elements may include vibrotactors.

Haptic devices 1410, 1420, 1504, and 1630 may include any suitablenumber and/or type of haptic transducer, sensor, and/or feedbackmechanism. For example, haptic devices 1410, 1420, 1504, and 1630 mayinclude one or more mechanical transducers, piezoelectric transducers,and/or fluidic transducers. Haptic devices 1410, 1420, 1504, and 1630may also include various combinations of different types and forms oftransducers that work together or independently to enhance a user’sartificial-reality experience. In one example, each of band elements1632 of haptic device 1630 may include a vibrotactor (e.g., avibrotactile actuator) configured to vibrate in unison or independentlyto provide one or more of various types of haptic sensations to a user.

FIG. 17A illustrates an exemplary human-machine interface (also referredto herein as an EMG control interface) configured to be worn around auser’s lower arm or wrist as a wearable system 1700. In this example,wearable system 1700 may include sixteen neuromuscular sensors 1710(e.g., EMG sensors) arranged circumferentially around an elastic band1720 with an interior surface 1730 configured to contact a user’s skin.However, any suitable number of neuromuscular sensors may be used. Thenumber and arrangement of neuromuscular sensors may depend on theparticular application for which the wearable device is used. Forexample, a wearable armband or wristband can be used to generate controlinformation for controlling an augmented reality system, a robot,controlling a vehicle, scrolling through text, controlling a virtualavatar, or any other suitable control task. As shown, the sensors may becoupled together using flexible electronics incorporated into thewireless device. FIG. 17B illustrates a cross-sectional view through oneof the sensors of the wearable device shown in FIG. 17A. In someembodiments, the output of one or more of the sensing components can beoptionally processed using hardware signal processing circuitry (e.g.,to perform amplification, filtering, and/or rectification). In otherembodiments, at least some signal processing of the output of thesensing components can be performed in software. Thus, signal processingof signals sampled by the sensors can be performed in hardware,software, or by any suitable combination of hardware and software, asaspects of the technology described herein are not limited in thisrespect. A non-limiting example of a signal processing chain used toprocess recorded data from sensors 1710 is discussed in more detailbelow with reference to FIGS. 18A and 18B.

FIGS. 18A and 18B illustrate an exemplary schematic diagram withinternal components of a wearable system with EMG sensors. As shown, thewearable system may include a wearable portion 1810 (FIG. 18A) and adongle portion 1820 (FIG. 18B) in communication with the wearableportion 1810 (e.g., via BLUETOOTH or another suitable wirelesscommunication technology). As shown in FIG. 18A, the wearable portion1810 may include skin contact electrodes 1811, examples of which aredescribed in connection with FIGS. 17A and 17B. The output of the skincontact electrodes 1811 may be provided to analog front end 1830, whichmay be configured to perform analog processing (e.g., amplification,noise reduction, filtering, etc.) on the recorded signals. The processedanalog signals may then be provided to analog-to-digital converter 1832,which may convert the analog signals to digital signals that can beprocessed by one or more computer processors. An example of a computerprocessor that may be used in accordance with some embodiments ismicrocontroller (MCU) 1834, illustrated in FIG. 18A. As shown, MCU 1834may also include inputs from other sensors (e.g., IMU sensor 1840), andpower and battery module 1842. The output of the processing performed byMCU 1834 may be provided to antenna 1850 for transmission to dongleportion 1820 shown in FIG. 18B.

Dongle portion 1820 may include antenna 1852, which may be configured tocommunicate with antenna 1850 included as part of wearable portion 1810.Communication between antennas 1850 and 1852 may occur using anysuitable wireless technology and protocol, non-limiting examples ofwhich include radiofrequency signaling and BLUETOOTH. As shown, thesignals received by antenna 1852 of dongle portion 1820 may be providedto a host computer for further processing, display, and/or for effectingcontrol of a particular physical or virtual object or objects.

Although the examples provided with reference to FIGS. 17A-17B and FIGS.18A-18B are discussed in the context of interfaces with EMG sensors, thetechniques described herein for reducing electromagnetic interferencecan also be implemented in wearable interfaces with other types ofsensors including, but not limited to, mechanomyography (MMG) sensors,sonomyography (SMG) sensors, and electrical impedance tomography (EIT)sensors. The techniques described herein for reducing electromagneticinterference can also be implemented in wearable interfaces thatcommunicate with computer hosts through wires and cables (e.g., USBcables, optical fiber cables, etc.).

The process parameters and sequence of the steps described and/orillustrated herein are given by way of example only and can be varied asdesired. For example, while the steps illustrated and/or describedherein may be shown or discussed in a particular order, these steps donot necessarily need to be performed in the order illustrated ordiscussed. The various exemplary methods described and/or illustratedherein may also omit one or more of the steps described or illustratedherein or include additional steps in addition to those disclosed.

The preceding description has been provided to enable others skilled inthe art to best utilize various aspects of the exemplary embodimentsdisclosed herein. This exemplary description is not intended to beexhaustive or to be limited to any precise form disclosed. Manymodifications and variations are possible without departing from thespirit and scope of the present disclosure. The embodiments disclosedherein should be considered in all respects illustrative and notrestrictive. Reference should be made to any claims appended hereto andtheir equivalents in determining the scope of the present disclosure.

Unless otherwise noted, the terms “connected to” and “coupled to” (andtheir derivatives), as used in the specification and/or claims, are tobe construed as permitting both direct and indirect (i.e., via otherelements or components) connection. In addition, the terms “a” or “an,”as used in the specification and/or claims, are to be construed asmeaning “at least one of.” Finally, for ease of use, the terms“including” and “having” (and their derivatives), as used in thespecification and/or claims, are interchangeable with and have the samemeaning as the word “comprising.”

What is claimed is:
 1. An artificial-reality system comprising: awearable dimensioned to be donned on a body part of a user, wherein thewearable comprises: a set of electrodes that detect one or moreneuromuscular signals via the body part of the user; and a transmitterthat transmits an electromagnetic signal; a head-mounted displaycommunicatively coupled to the wearable, wherein the head-mounteddisplay comprises a set of receivers that receive the electromagneticsignal; and one or more processing devices that: determine, based atleast in part on the neuromuscular signals detected via the body part ofthe user, that the user has made a specific gesture; and determine,based at least in part on the electromagnetic signal, a position of thebody part of the user relative to the set of receivers when the usermade the specific gesture.
 2. The artificial-reality system of claim 1,wherein at least one of the processing devices is incorporated in thewearable.
 3. The artificial-reality system of claim 1, wherein at leastone of the processing devices is incorporated in the head-mounteddisplay.
 4. The artificial-reality system of claim 1, wherein: thewearable comprises a first short-range radio; and the head-mounteddisplay comprises a second short-range radio that is communicativelycoupled to the first short-range radio, the first and second short-rangeradios being configured to exchange configuration data between thewearable and the head-mounted display.
 5. The artificial-reality systemof claim 4, wherein the first and second short-range radios are furtherconfigured to exchange data about the neuromuscular signals between thewearable and the head-mounted display.
 6. The artificial-reality systemof claim 5, wherein at least one of the processing devices generates,based at least in part on the data about the neuromuscular signals, aninput command that causes the head-mounted display to modify at leastone virtual component to account for the specific gesture.
 7. Theartificial-reality system of claim 1, wherein the transmitterincorporates a time stamp into the electromagnetic signal beforetransmitting the electromagnetic signal to the set of receivers.
 8. Theartificial-reality system of claim 7, wherein at least one of theprocessing devices: determines a first time of arrival for theelectromagnetic signal as received by a first receiver included in theset of receivers; determines a second time of arrival for theelectromagnetic signal as received by a second receiver included in theset of receivers; and calculates, based at least in part on the firstand second times of arrival for the electromagnetic signal and the timestamp, an angle of arrival for the electromagnetic signal relative tothe set of receivers.
 9. The artificial-reality system of claim 8,wherein at least one of the processing devices: calculates, based atleast in part on the angle of arrival, at least one dimension for aposition of a virtual component within a field of view of thehead-mounted display; and presents the virtual component at the positionwithin the field of view of the head-mounted display based at least inpart on the dimension.
 10. The artificial-reality system of claim 9,wherein the dimension calculated for the position of the virtualcomponent comprises at least one of: an azimuth for the virtualcomponent to be presented within the field of view of the head-mounteddisplay; an elevation for the virtual component to be presented withinthe field of view of the head-mounted display; or a depth for thevirtual component to be presented within the field of view of thehead-mounted display.
 11. The artificial-reality system of claim 9,wherein at least one of the processing devices: determines a first phaseof the electromagnetic signal as received by the first receiver includedin the set of receivers; determines a second phase of theelectromagnetic signal as received by the second receiver included inthe set of receivers; and calculates, based at least in part on adifference between the first and second phases of the electromagneticsignal and the time stamp, the angle of arrival for the electromagneticsignal relative to the set of receivers.
 12. The artificial-realitysystem of claim 9, wherein at least one of the processing devices:calculates, based at least in part on the angle of arrival, atwo-dimensional position for the virtual component within the field ofview of the head-mounted display; and presents the virtual component atthe two-dimensional position within the field of view of thehead-mounted display.
 13. The artificial-reality system of claim 9,wherein at least one of the processing devices: calculates, based atleast in part on the angle of arrival, a three-dimensional position forthe virtual component within the field of view of the head-mounteddisplay; and presents the virtual component at the three-dimensionalposition within the field of view of the head-mounted display.
 14. Theartificial-reality system of claim 9, wherein: the virtual componentpresented at the position comprises a pointer presented at the position;and at least one of the processing devices superimposes the pointer overa screen of the head-mounted display.
 15. The artificial-reality systemof claim 14, wherein at least one of the processing devices generates,based at least in part on data about the neuromuscular signals, an inputcommand that causes the head-mounted display to modify at least oneadditional virtual component presented proximate to the pointer withinthe field of view of the head-mounted display to account for thespecific gesture.
 16. The artificial-reality system of claim 9, whereinat least one of the processing devices: determines, based at least inpart on the angle of arrival, that the wearable is no longer visiblewithin the field of view of the head-mounted display; and in response todetermining that the wearable is no longer visible within the field ofview of the head-mounted display, removing the virtual component fromthe field of view of the head-mounted display.
 17. Theartificial-reality system of claim 1, further comprising an additionalwearable dimensioned to be donned on an additional body part of theuser, wherein the wearable comprises: an additional set of electrodesthat detect one or more additional neuromuscular signals via theadditional body part of the user; and at least one additionaltransmitter that transmits an additional electromagnetic signal;wherein: the head-mounted display is also communicatively coupled to theadditional wearable, wherein the set of receivers receive the additionalelectromagnetic signal transmitted by the additional transmitterincluded on the additional wearable; and at least one of the processingdevices: determines, based at least in part on the additionalneuromuscular signals detected via the additional body part of the user,that the user has made an additional gesture; and determines, based atleast in part on the additional electromagnetic signal received by theset of receivers included on the head-mounted display, a position of theadditional body part of the user when the user made the additionalgesture.
 18. The artificial-reality system of claim 17, wherein at leastone of the processing devices: calculates, based at least in part on theelectromagnetic signal, at least one dimension for a position of avirtual component within a field of view of the head-mounted display;calculates, based at least in part on the additional electromagneticsignal, at least one additional dimension for an additional position ofan additional virtual component within the field of view of thehead-mounted display; and simultaneously presents, within the field ofview of the head-mounted display, the virtual component at the positionand the additional virtual component at the additional position based atleast in part on the dimension and the additional dimension.
 19. Ahead-mounted display comprising: a set of receivers configured toreceive an electromagnetic signal transmitted by a transmitter includedon a wearable dimensioned to be donned on a body part of a user; a radioconfigured to receive data about one or more neuromuscular signalsdetected by the wearable via the body part of the user; and at least oneprocessing device communicatively coupled to the set of receivers andthe radio, wherein the processing device: determines, based at least inpart on the data about the neuromuscular signals, that the user has madea specific gesture; and determines, based at least in part on theelectromagnetic signal, a position of the body part of the user relativeto the set of receivers when the user made the specific gesture.
 20. Amethod comprising: detecting, by a wearable donned on a body part of auser, one or more neuromuscular signals at the body part of the user;emitting, by a transmitter included on the wearable, an electromagneticsignal; receiving, by a set of receivers included on a head-mounteddisplay donned by the user, the electromagnetic signal emitted by thetransmitter; determining, by one or more processing devices, that theuser has made a specific gesture based at least in part on theneuromuscular signals; and determining, by the one or more processingdevices, a position of the body part of the user when the user relativeto the set of receivers made the specific gesture based at least in parton the electromagnetic signal.